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The project objectives were: (1) to assess existing 
industrial upgrading practices in an Atomic Energy Commission 
contractor organization, (2) to design new alternative upgrading 
methods, (3) to experiment with new upgrading methods, m to plan 
for utilization of proven upgrading programs, and (5) to document and 
disseminate activities. A twelve page section of the report Is in the 
form of a separate paper, titled The Validity and Utility of 
Simulation Techniques, discussing the operation of an assessment 
center for employee upgrading. A further 30 pages appear to be from a 
booklet explaining how the assessment center is used for 
identification of first-line supervisors. Six pages summarize the 
impact of management practices, training programs, legislation, 
collective bargaining agreement, and employee mobility on employee 
upgrading* The final three pages briefly evaluate the success of the 
assessment cantar and the opportunity counselling it provided. 
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INTRODUCTION 

The format of this report is intended to provide for the maximum use 
of the written outputs of the study by different audiences. Segments of 
the report are printed as independent brochures targeted to specific 
audiences according to interest. These brochures are designed as utiliza- 
tion materials. 

This final report is submitted to the Office of Research and Develop- 
ment, United States Department of Labor by Manpower Development Division, 
Oak Ridge Associated Universities (MOD) in fulfillment of contract 
conditions. In this form the report will have limited distribution. 
Parts of the report are published for much broader distribution, notably 
the brochure explaining the assessment center approach for the identifica* 
tion of management potential. The brochure lists other publications in 
this report that will be available separately upon request. This somewhat 
unusual strategy is designed to support both the fourth and Sixth objec-^ 
tives of the project (below), the utilization of proven upgrading programs 
and the liocumentation and dissemination of activities. 

Ail materials in this report were generated as part of the study of 
employee upgrading in an industrial setting. The project was initiated 
on October 1, 1971, and ended December 31, 1973. During this 27-month 
period, three types of research and development activities were conducted: 
(1) data gathering and analysis, (2) applied experimentation, and 
(3) utilization. 

The project objectives were: 

1. to assess existing industrial upgrading practices in an 
Atomic Energy Commission (AEC) contractor jorganizat ion 
in order to identify those factors that significantly 
influence the rate and amount of upgrading; 

2. to design new alternative upgrading methods developed 
with the effective participation of management, union, 
MDD staff and individual employees aimed at improvement 
of upgrading efforts; 

3. to experiment with at least three new upgrading methods; 

4. to plan for the utilization of proven upgrading programs 
in additional AEC contractor organizations; and 

5. to document and disseminate activities. 
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The accomplishment of the first objective is documented in the re- 
port of Phase I, Part 1, which is summarized in Section I of this report. 
This analysis was intended to lead Manpower Development Division and the 
cooperating industry to conclusions about the current upgrading system 
and to suggest new alternatives that could be tested. Submission of the 
first report to the company met with mixed reactions. 

At top management levels^ support was found for continuation of the 
project directed at accomplishing the second and third objectives above. 
However, management wanted assurances that the project would not disrupt 
routine work efforts. One of the company's three plants who participated 
in the study at the outset was willing to implement procedures and practices 
that required change. 

For some time prior to MDD's intervention in the company* s upgrading 
system, certain top level managers were concerned with selecting first- 
line supervisors. Their concern had resulted in the implementation of an 
experimental assessment center. MDD found the assessment center approach 
to be infrequently used and lacking in certain respects. As originally 
used in the plant, an assessment center session was only convened when a 
promotion to first- line supervisor was available. Candidates were highly 
screened before participation in the center, and very few sessions had 
ever been conducted. The procedures for assessing a candidate's perfor- 
mance were, lengthy and clinical in character. This situation, in conjunc- 
tion with the projection that this plant was about to undergo a sizable 
expansion, appeared to be the basic reason this plant supported the second 
objective, to design new upgrading methods from a basis of cooperation 
among management, union, upgrading project staff and individual ^ployees. 

MOD and plant personnel determined that modifications in the assess- 
ment center approach could have two immediate positive effects. If a 
reduction in the length of time required to conduct the center sessions 
could be accomplished, more candidates could be processed. Such a change - 
was important since promotional opportunities were forecast in the near 
future for a large number of employees. Another implication of reducing 
the time required was a concomitant reduction in cost per candidate. MDD 
staff recognized that if efficiency in scoring could be realized, time 
could be reduced. The character of clinical discussion which required a 
great deal of staff time was viewed as the one area in which time reductions 
could be made without sacrificing quality. , 
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Modification of the scoring system also permitted the staff to address 
another serious shortcoming of the procedure as i t then existed, its lack 
of demonstrable objectivity. This problem could be approached concurrently 
with the one of time-efficiency by designing the new scoring procedures 
which would better follow principles of psychological measurement and that 
would yield data to eventually answer questions of validity.^ 

The company ^s management was also aware of a morale problem among one 
division's technicians, MDD's proposal to alleviate the problem was 
accepted by company management, and entailed making available opportunity 
counseling to employees after their participr^tion in the assessment center. 
The counselor used information about the employee gathered in the assess* 
ment center, information from an interest inventory and other paper- 
pencil aptitude tests, and information about job openings. During the 
counseling sessions, an interview was conducted with each employee. The 
purposes of the interview were to get each individual's reaction to the 
diagnostic process, to explain the information collected about each 
employee, to describe job opportunities, and to explore alternative 
development plans. 

s. 

Unexpectedly, opportunity counseling was also viewed by current 
managers as a tool for solving some of their own problems. Managers often 
visited the opportunity counselor to discuss ways to maximize utilization 
of anployee's talents and to explore solutions to problems of interpersonal 
relations in their work. These discussions were usually focused on 
individual employees 

MDD staff assumed at the beginning of its work with the cooperating 
company that a need existed for quite extensive employee upgrading. This 
assumption was not confirmed by the examination of the existing upgrading 
system. The company's needs, and to a ]^arge extent the employees' needs, 
were being met. Low employee turnover and Tionexpanding work force were 
the two primary circumstances that worked against implementation of major 
training efforts. 

To accommodate the study^s third objective, MDD's proposed experimen- 
tation with new upgrading methods, the company agreed to increase its use 
of the assessment-center. This provided larger numbers of employees the 
opportunity to participate, and allowed employees to nominate themselves 
for promotional consideration. Also, the company agreed to implement 
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formalized opportunity counseling as an individualistic approach to 
employee upgrading. 

Section I of this report contains a summary of an interiir. report 
describing the activities and rtsults of MDD's intervention in the com- 
pany's upgrading activities. In part, the second and third objectives of 
this project were accomplished. Although three approaches were not tested, 
the most reasonable intervention strategy was.' 

The study's fourth objective, to obtain AEC network utilization of 
the promising upgrading strategy, was planned to occur during a second 
phase that was not funded. However, unexpended funds from Thase I wcio 
used to explore interest in employee upgrading at other AEC construction 
sites. Subsequent to discussions with industrial relations personnel at 
the AliC-V/ashington Headquarters^ a survey of select ie^i contractors was 
initiated. Interest in upgrading was expressed by 18 contractors, and a 
determination to conduct a utilization workshop on employee upgrading 
was made. 

The workshop was held in Oakland, California, at the AEC area 
operations office. Thirteen of the 18 companies expressing interest were 
represented by either their respective industrial relations director, 
personnel director, equal employment opportunity director, or training 
director. Presentations and documents prepared for the workshop are 
included in Section II. 

In summary, MUD studied existing upgrading practices in a three- 
plant company; the need to suggest major training efforts did not exist 
since the preparation for promotions was either accomplished via appren- 
ticeship programs or on-the-job training, and advancement opportunities 
were limited by the exceptionally stable labor force. 

Using a theoretical model against which to compare actual company 
practice, qertain disparities were found. Pragmatically, however, these 
disparities were not perceived by management as severe enough to warrant 
major change. Therefore, minor experimental activities were conducted 
on a pilot basis. In fact ^ these minor activities (the assessment center 
and opportunity counseling) served to correct a disparity in how super- 
visory candidates were selected, and has increased the pool of applicants 
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FRJ^C Industrial Setting , Oak Ridge Associated Universities, Oak Ridge, 
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available for management consideration. Adoption of these two components 
of an upgiTading system has. been accomplished because they helpwu to fill 
a company need, and they are now gaining greater management acceptance. 

Utilization in other companies should not be expected from a single 
workshop because the adoption of this small portion of an upgrading system 
is still very complex and requires a considerable investment of time and 
money. Furthermore, unless the adopting fiKm recognized selection of 
first- line supervisors as a pressing need, very little company action 
could be anticipated. 

The utilization workshop confirmed a growing management perception 
of the need to move women ^d minorities into management positions. The 
assessment center and opportunity counseling approach can help in accom- 
plishing that objective if the perceived need becomes sufficiently 
demanding. 
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VALIDITY and UTILITY 





THE VALIOiTY AND UTILITY OF 
SIMULATION TECHNIQUES 



Paper presented at 



.f * f y-'^^.T^ -rrv-/; v';t.'v- \\'>^'V>'<,?A' 



Utilization Workshop for 
Employee Upgrading 



San Francisco, California 



Manpower Development Division 
Oak Ridge Associated Universities 



THE VALIDITY AND UTILITY OF 
SIMULATION TECHNIQUES 



' Manpower Development Division staff looked 
closely at the operation and scoring of several 
simulation techniques. One of the most common 
questions asked is whether the method is valid. ' 
Actually, there ^'re many questions which must be 
answered to demonstrate the validity of simula- 
tion techniques. The most important ones are 
those concerning the relationship of exercise 
results and external criteria of job success or 
predictive validity. The earliest and strongest 
evidence for predictive validity comes from the 
work done by Bray and Grant with several hundred 
new hires into AT&T.^ These people were assessed 
using simulation techniques and then followed up 
several' years later without the data being 
allowed to influence their careers in any way. 
The rater's predictions about who would move into 
middle management ranks were accurate and signi- 
ficantly above the chance level. There have been 
a whole series of validity studies done by the 
Bell System and other large corporations which 
employ the assessment center procedure. Findings 
from these studies further substantiate the 
validity of simulati^on techniques. In a recent 
article which reviewed assessment literature and 
focused on the predictive accuracy of the overall 
assessment ratings (a global varicible which is 
common to almostr'^ll assessment programs), 18 
research studies conducted from 1964 to mid-1972 
consistently showed assessment performance re- 
lated to several external criteria. The predic- 
tive accuracy of overall assessment rating was 
highest for job potential followed by progress in 
management or number of promotions and then job 
performance 

The findings of these studies were replicated 
in a followup of early participants in an assess- 
ment program under MDD's sponsorship with a 
cooperating company. Significant correlations 
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were found between the overall rating and the 
current supervisor's ratings of performance and 
potential. Additionally, a high correlation was 
found with a number of promotions sinct the date 
of their assessment • Thus, those rated high in 
the assessment center had significantly more pro- 
motions since the date of their assessment. 
Thus , those rated high in the assessment center 
had significciutly n,ore promotions and higher 
criterion ratings than those rated low.^ 

Most of the research evidence seems to indi- 
cate that the assessment center produces a modest 
improvement in performance at the first level, 
but the differences in results for performance 
and potential seem to suggest that the management 
skills measured at the assessment center are more 
important in higher levels of supervision. This 
may be accounted for by the fact that lower level 
supervision jobs involve more technical functions 
and less organization and management. 

Thus, one is led to conclude tentatively 
that simulation techniques have validity in pre- 
dicting who will move ahead in an organization. 
Most of the studies have flaws, but there is a 
consistent pattern of apparent validity. This 
does not imply that all simulation techniques 
should be considered valid. Any organization 
adopting a simulation approach should establish 
em^^irical procedures to evaluate it. Relation- 
shops between measurements obtained in the 
exercises and external criteria of job effec- 
tiveness must be thoroughly investigated. 

With the increasing enforcements of EEOC 
and OFCC guidelines , organizations must be pre- 
pared to offer proof that their selection and 
appraisal devices including simulation techniques 
are valid and job related. Perhaps because of 
the "rational validity" which results from the 
situational nature of the exercises no known 
charges of discrimination from application of an 
assessment center have been filed anywhere in 
the U. S. 
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The few studies that have attempted to col- 
lect empirical evidencfj of the validity of the 
approach for minority group members have found 
positive results. One author examined both one- 
day and three-day assessment programs at AT£T 

and reported that no differential validity re- . . v'.. ..v*,. ■; ... .\ 

suits were found between any of the SLt»groups ^^■'''^/^'^rf^^ 

based on race dn'^/or sen.'* The only other ' " . '• " ' 

published study of differential validity dealt 

with assessment centers at General Electric. Of 

120 whites assessed, 25% received job offers; 

while of 30 blacks assessed, 50% received offers.^ 

Thus , no bias in favor of whites appeared to be 

operating. However, validity in one organization 

does not necessarily imply that the procedure is L, t. . * --V - ':'>!t;^\v. 

valid in another, so evidence should be collected '^■4::.%,/tAvw«'^ . : • " '*•*,^-••A^^^^' ' 

to evaluate the fairness of any assessment procedure. / . 



Important questions as to the validity of 
simulation techniques exist at another levels 
that of face validity, that is, the acceptability 
of the procedure to the candidates and to manage- 
ment. For example, it would certainly be an 

undesirable consequence if low-rated candidates f ^•^(^•^• tfij^'W'-Vpi'Vr'ii^V'. V;*^ 

were sufficiently demoralized to leave the com- ; 
pany, as this would result in the loss of well- ;* ..;/ >: '' "[.■^^^^^^^^^ 

trained, adequately performing employees. jr" ^'^i^' 

(Incidentally, the followup of 1,000 assessment 
center candidates at IBM showed no difference in 
separation rates for low and high assessed 
employees.^) One phase of evtiluating the -cooperat- 
ing company's assessment program was a survey of 

the people it served (managers, supervisors and .... 
participants). The results of the survey are . : . ^ \.- *• * • . "\iv - 

comparable to the experiences of many organiza- " ' ' ' 

tions . 



The first part of the survey asked super- 
visors and department heads who had extensive 
contact with the program to complete a brief 
questionnaire pertaining to the center's general 
utility and to problem areas The 32 respon- 
dents to date have all felt that the assessment 
center approach provided useful information. 
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Table 1 shows the role that assessment center 
results play in promotional . decisions - It is 
q.uite apparent that while few supervisors or 
department heads rely solely on the exercise re- 
suits for promotional decisions, most do take 
tlie recommendations quite seriously. 



TABLE 1 




Utilization of Assessment Center 
Results by Managers'- 




Assessment center the main factor 
in promotional decisions 


12% 


Assessment center a major factor, 
but ^technical competence more 
important ^ 


23% 


Assessment center will be a strong 
factor, but not the major one 


50% 


Used in borderline cases 


15% 


Not used for promotional decisions 


0 


•'^N = 32 . 





As well as being used in making promotional 
decisions, the assessment center results are 
being extensively used -for counseling and employee 
development. In fact, quite a few of the super- 
visors and managers felt that this is the area 
in which the results were most useful. 

The candidates ' immediate supervisors were 
also asked to respond to questions about the 
assessment center's accuracy and usefulness for 
eacn employee's particular case (TaJDle 2). In 
bl'o of the cases, 31 of 51, the supervisors re- 
ported the assessment center was generally 
accurate. While eight percent thought it was 
extremely accurate, 14% also thought it was not 
accur;^te at all. The reason for this slightly 
negative attitude seems to be that some supervisors 
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TABLE 2 

Supervisors * Report of 
of Assessment Cente 

Extremely accurate 

Generally accurate 

Partially inaccurate 

Not accurate all 



••^N = 51. 



saw some discrepancy between the candidate's job 
performance and the assessment center reports, 
although this may certainly be expected because 
there are differences in the behaviors exhibited 
on their current jobs and the behaviors they 
were asked to exhibit in the assessment proce- 
dure. The most important criticism came from 
some of the supervisors who reported that the . 
exercises were not as relevant for their depart- 
ment: as they were for others; hence, they felt 
their employees might be at a slight disadvantage. 
The implication is that the simulation exercises f*'* •^H'''^^^• i;-r'^vr?*"''f<^<\:f<J ^ft^^^ 

simulated some jobs better than others ► This is -v . : * . 

certainly a problem which must be contended with '. . \ " • . ' .'**' *. 

in any organizational setting in which simulation '.^iJ^ — : — '—lj- i ■ ■ * ' 

techniques are used to. assess candidates from a 
wide variety of jobs. Resear.ch indicates that 
the assessment center can be made flexible enough 
so that it is valid for a variety of jobs* 

Participants in the program were also siar- 
veyed concerning their impressions of the experi- - 

ence, fifty-seven percent participated to date. -1 * • ' " * *»• 

Ninety percent of these participants believed 
the assessment center procedure to be fair, and 
S7% felt that it allowed a chance to show their 
reel capabilities for supervisory positions. 
Wi ti; respect to the report which the manpower 
planning stafr' prepared for the participants' 
department , Table 3 summarizes the opinions 



• the Accuracy 
ir Results^ 



61% 
17% 
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concerning th^^. iccuracy of the report. Thus, one 
can see that v*?ry few of the participants felt 
the results were inaccurate. 



TABLE 3 




Participants' Estimates of Accuracy 
of Arsessment Center* 




Extremely accurate 


2% 


Very accurate 


47% 


Fair 


40% 


Inaccurate 


9% 


rotally inaccurate 


2% 


= 45. 





An important purpose of the followup of the 
participants was to assess the short-term impact 
that the assessment center had on their percep- 
tion of the company and themselves (Table 4). 
Most employees seemed to have a generally 
favorable attitude toward their employer both 
before and after participation. One reported a 
less favorable attitude, while 28% reported a 
more favorable one. 

Did participation in the assessment center 
cnange their attitudes about themselves? Forty- 
four percent claimed a more favorable attitude, 
and 48% claimed no change, while eight percent 
saia they have a less favorable attitude toward 
themselves as a result of their disappointing 
performance in the center. Most employees re- 
ported that they gained increased self-confidence 
plus some insight and a more realistic view of 
their capabilities. As a result of the increased 
attention being given them by the company, most 
of the participants reported that their chances 
for promotion had increased. 

One outstanding result of the evaluation 
process is the fact that most employees now have 
a clear idea of their chances for advancement. 



TABLE k 




Impact on Attitudes 




Attitude Toward Employer^ 




More favorable 


28% 


No change 


72% 


Less favorable 


0% 


Attitude Toward Themselves^ 




More favorable 


4^% 


No change 


48% 


Less favorable 


8% 







This is shown by the high correlation between 
their estimates and their supervisor's estimates 
of their chances for promotion. The correlation 
was •Bl, which is highly significant. 



These results and others obtained in the, 
survey tend to demonstrate that the simulation 
procedure has a very high acceptance and face 
validity among the participants, supervisors, 
and managers involved with the assessment center. 

Two other levels of validity must also be 
considered. One is the utility of the techniques, 
that is, the advantage of using them over tradi- 
tional and/or less costly methods; and the other 
is the identification of the performance dimen- 
sions and the reliability of the measurement 
procedures. These levels are concerned with a 
type of internal validation. 

One unique feature of the assessment center 
procedure is the use of multiple observers or 
assessors whose judgments regarding the observed 
performance of candidates are pooled. This 



clinical type of pooling makes questions of r^alia- 
bility somewhat more difficult to answer, since 
agreement between raters is reached before final 
ratings are recorded • 

A study conducted by Michigan Bell provides 
the mpst definitive data regarding reliability 
of the total assessment process,^ The study was 
designed to measure reliability of candidates' 
performance over tvo assessment programs • A 
sample of 85 non-management employees (39 men, 

women, blacks, and 43 whites) who attended 
an Early Identification Assessment program were 
later assessed by the coiripany's more extensive 
two-day assessment program. The minimum time 
between assessments was one month and partici- 
pants in the study did not receive any feedback 
on their performance until after completing both 
programs • Independent assessment staffs were 
used« 

The correlation between overall performance 
in the two programs was quite substantial for 
the total sample (173), as well as for each of 
the subgroups : 

Men - .77 Blacks - .ee 

Women - ,70 Whites - ,73 

This is one of the few studies which deals with 
consistence of assessees ' performance over time. 

Besides the overall rating all of the speci- 
fic variable ratings collected in the two assess- 
ment programs showed significant reliability. 

When contrasted with the relative invisibi- 
lity of many existing promotional systems, it is. 
apparent that assessment programs are more likely 
to encourage decision' making about promotions to 
be made openly and objectively with agreed-upon, 
standards and based on relevant systematically 
gathered data. 



An important question is whether the less 
costly procedure of using standardized tests can 
be as effective in prediction as simulation 
techniques • The research to date indicates that 
simulation results cannot be replaced by tests 
but also that the use of simulation does not 
negate the use of tests, that is, they each may 
contribute substantially unique elements to the 
prediction of success.^ This leads to an impor- 
tant point. Situational techniques are designed 
to provide valid and job related information; 
however, there are many areas of job performance 
which these techniques may not reach. It has 
already been pointed out that technical compe- 
tence is not measured by these exercises, and 
this may be particularly important for first-line 
supervision. Similarly ^x^certain aptitudes may be 
important for successful job performance which 
can best be measured by validated tests . 

Existing methods of measuring on-the-job 
performance can provide much useful information 
on technical competence and on other aspects of 
performance by an employee, who may have a job 
history in areas similar to the position for 
which he is being considered. The point is in 
comparing situational techniques to other promo- 
tional methods, it is not a matter of determining 
which method is the. best but rather a determina- 
tion should be made on the basis of the value 
added by each method. 

When simulation techniques have been compared 
with paper and pencil tests used for the same pur- 
pose, they have consistently explained more 
variance on the criterion. When simulation tech- 
niques have been compared to sundry techniques 
used independently of the assessment center, the 
assessed group was found higher in job performance 
and progress in management. ^ Thus, the conclu- 
sion of more researchers to date is that simula- 
tion techniques do provide unique and valuable 
information, p^LVtLcularly in the field of 
interpersonal behavior. 



-10- 



FOOTNOTES 

!• Bray, W, and Grant, "The Assessment 

Center in the Measurement of Potential for 
Business Management," Psychological 
Monographs ; 1966, GO (17, whole no. 625). 

2. Cohen, B. M, , Moses, J. and Byham, W. C. 
"The Validity of Assessment Centers: A 
Literature Review." In Klinefelter J. 
"Assessment Center Approach: A Comprehensive 
Summary*" Division of Research and Test 
Development, State of Illinois, Department 

of Personnel. 

3. 'Internal Research Document, Oak Ridge 

Associated Universities, Manpower Development 
Division. 

4. Moses, J. C. The Development of an Assess- 
ment Center for the Early Identification of 
Supervisory Potential. Personnel Psychology 
(in press) . 

5. Meyer, H. H. "Assessment Centers at General 
Electric." Paper presented at meeting of 
Development Dimensions, San Francisco, 
California, 1972. 

6. Cited in: Kraut, A. I. "A Hard Look at 
Management Assessment Centers and Their 
Future." Personnel Journal , May 1972, 
317-326. 

7. "Upgrading in an Industrial Setting, Phase I, 
Part 2: A Pilot Test of Upgrading Models." 
Oak Ridge Associated Universities, Manpower 
Development Division. 




8. Moses, Op Cit . 



Huck» James "Assessment Centers: A 
Review of the External and Internal Validi- 
ties/* Personnel Psychology , V. 26, 1973, 
191-212 • 



10 • Cohen, Op Cit . 



.V > .►'•J •^J..1^.• 



ERIC 



Simulation Techniques 

• Needs to be validated 

• May be misused 

• Offers opportunity to observe and assess behaviors difficult 
to assess with other techniques 

• May be more expensive than other approaches 
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ASSESSMENT CENTER: 
OPERATION 
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ASSESSMENT CENTER DEFINITION 

An assessment center is a multiple assessment of several 
individuals performing a variety of group and individual 
exercises simulating work situations while their behavior is 
observed by a group of trained evaluators. 



ASSESSMENT CENTER-PROGRAM OBJECTIVES 

• Early and uniform selection of future supervisors. 

• Supervisory training tailored to predetermined needs. 

• Continuous training and development of new front*line 
supervisors. 

• Provision of well-trained and motivated supervisory 
personnel to fulfill present and future plant needs. 
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AT&T in the early 50*$ pioneered the use of Assessment 
Center in the selecticp of management J Their success lead to 
relatively widespread use. 

Companies now using Assessment Center: 

ATfciT-Bell System 

Standard Oil of Ohio 

J. C. Penney Company 

Peace Corps 

Caterpillar Tractor 

Sears Roebuck and Company 

IBM 

General Electric Company 

IRS 

TVA 

Otin Mathieson Chemical Corporation 

V/olverine Tube Company 

Wickcs Corporation 

Union Carbide-Nuclear Division 

Ford Motor 

Zenith 

Jewol Foods 

Federat Aviation Agency 



^ 3rjy Cffnt 
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This booklet*fexplains how the Assessment Center is used for 
identification offirst*line supervisors. 




Approaches to selection now in use. 

• Supervisory nomination I* 

• Standardized (paper pencil) tests 

• Personality tests 

• Acting assignments 

• Simulation techniques 
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Supervisory Nomination 

• May not be based on relevant job information 

• Is not standardized across interviewers 

• Personal preferences may inject biases 
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Standardiied (Paper -Pencil) Tests 

• Correlates moderately with managerial success 

• Frequently subjected to misuse and misinterpretation 

• Too often not validated on performance criteria 

• Inexpensive to admiffister 
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Personality Tests 

• Has not been shown to have strong relationship to 
managerial success 

• Frequently subjected to misuse and misinterpretation 




What do you see? 



I.' 
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Acting Assignments 

• Not always recognized as an upgrading technique 
0 May be misused 

• Failure may permanently disqualify employee 
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Simulation Techniques 

• Needs to be validated ' 

• May be misused > 



• Offers opportunity to observe and assess behaviors difficult 
to assess with other techniques 

• May be more expensive than other approaches 



1M»II IV UVI«P«HI 
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Design of an Assessment Center Program 

Answer these questions: 

What does a manager do? 

What are the skills a manager needs? . 
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Design a procedure which will reveal the behaviors you look 
for. Create situations where individuals can exhibit behavior 
indicative of these skills. 



Decide who shall be evaluated. 



EXAMPLE OF AN ASSESSMENT CENTER PROGRAM 
Assessment Center Exercises 

• Leaderless Group Discussions 

—Assigned Role— School board exercise candidates adopt 
particular, competitive roles 

— Nonassigned Role— Community planning problem, 
management consultant problem 

Candidates choose own role and cooperate to achieve a 
common goal. 

• Individual Exercise— ln*Basket 
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Assessment Center Operations-Sequence of Events 

• Referrals of candidates 

• Prepanel interview of candidates 




SUPERVISOR'S REPORT 



Date of Interview 
May 16> 1 973 



Employee's Name . Badge No. Department No. 

Clarence Clyde McWiHiams 38291 1079W 

Reason 

Recommendation for Supervisory Panel Evaluation 

Details 

CSD: 6-17 69 

Birth Date: 2-2447 



Marital Status: Single 



Address: Hlllcrest Apartments, Apt. 47 
Oak Ridge, Tenmssee 37830 

Education: Graduate of Elm^r G. Smith College 

Mr. McWilllams received a bachelor of arts degree in economics from 
the Elmer G. Smith College of Rolling Fork, Mississippi, in May 

1969. He yyas a candidate for Cum Laude honors and was on the 

Dean's List five of seven semesters. 

He was employed with the Company on June 17, 1969, as a Buyer in 
the Purchasing Division. He transferred to the Maintenance Engineering 
Department on July 6, 1970, and has been serving as an Engineering 
Assistant. 

Mr. McWiliiams has conducted studies and prepared reports for the 

exterior and interior paint programs in the plant. He has made recom- 

mendatlons which have been accepted and are being Implemented to 

[mprove the painting operation and decrease the total cost. He is 

responsible for updating and preparing new Maintenance Engineering 

Procedures and performs other assignments as required. 

Mr. McWiliiams is actively engaged in the improvement of our mainte- 
nance program through implementation of our Mgintenance Manage- 
ment Control System. 

Employee's Acknowledgement Supervisor's Signature 

of Receipt of Copy 



Distribution: Employee Relations Department 

File 

Others as Required 
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Pre^Panel Interview 



At this interview, the department head will describe the 
program to the employee and verify his interest. He will be 
advised that it affords an excellent opportunity to recognize 
individual development needs as observed by six experienced 
supervisors. If the employee is interested, it is essential that, 
before he is allowed to go before the panel, he be advised of 
the following: 

I 

- That his participation is entirely voluntary. 

- That his participation in the prociram wilt not guarantee 
job advancement. 

- That the results of his participation in the program will not 
have a lessening effect on his current position. 

— That information compiled by the panel is considered 
Business Confidential. It will be made a part of his 
personnel record; but due to its confidentiality, it will be 
accessible to a limited number of management personnel. 

— That when called before the panel (panels are scheduled 
irregularly), he will be participating in a full day of group 
and individual exercises with other employees. 

— That his performance throughout the day will be closely 
observed and evaluated by a panel of six experienced 
supervisors. 

— Thai he will be informed of the results of his performance 
through his line organization upon his request. The 
information will f)C given with the objective of assisting 
the candidate in his stMt d -^elopmeni. 

- Thai the exercises are ues* jned lo measure administrative 
and supervisory skills; they are not designed to determine 
technical competence in any vocational area. 

-31- 



Screen and select six candidates and six trained management 
observers. 



Typical Assessment Cenf'^r Time Schedule 

8:30-9:30 Exercise 1, Cdmmunitv Planning * 

r ■ 

9\30-10:15 Exercise 2, School Board-Planning Period 

10:15--10:20 Break 

10:20-1 1 :20 Exercise 2, School Board 

11:20-12:00 Lunch 

12:05-2:05 Exercise 3, In Basket 

2:05-2:20 Break 

2:20-2:45 Exercise 3, In-Basket Interview 

2:45-3:50 Exercise 4. Managemerit Consultant Problem 

3:50-4:00 Final Interview 

4 00-4:15 Panelist Wrap-Up • 
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SEATING AND OBSERVATION ARRANGEMENT 
Candidates Positions 




• Sealing and observation arrangements assist in effective 
reporting of behavior. 

• Candidates change positions during the day. 

• Panelists complete a behavioral checklist following each 
Leaderless Group Discussion 

• Panelists, only, meet on the second day to critique each , * 
candidate's performance. 

• Primary observer explains his observation-other panelists 
add their comments, 

• Numerical ratings are assigned on each variable for every 
candidate. 

• An overall rating is given ;or each candidate's performance. 

• A Program Coordinator, using evaluations and behavioral 
checklists, writesa final evaluation report. 



ASSESSMENT CENTER PROGRAM 
FINAL EVALUATION SUMMARY 



Candidate 



Date 



Division 



Panel Session 



Rating Scale of 
Rating Candidate PerfornDance 



INFLUENCE OR LEADERSHIP- 
Demonstrates the ability to plan, 
organize, and control the group 
process. 



BEHAVIORAL FLEXIBILITY- 
Demonstrates the ability to 
accommodate arguments and 
behavior to the achievement of 
gr^iup objectives. 



RELATIONSHIPS WITH PEERS- 
The judged relationships formed 
during group exercises between the 
individual and other group 
members. 



ORAL COMMUNICATION 
SKILLS^Demonstrates effective 
delivery; ts easily understood by 
others. 



ACTIVITY-Judged on the amount 
of participation in group exercise 
and on productivity in the written 
exercise. 



PERCEPTION-Dcmonstrates 
recognition of problems in the 
exercise, along with an under* 
standing of other candidates' 
arguments and positions, and 
shows insight into the interrela* 
tionships of these factors. 



EXCELLENT --Performance in 
the Assessment Center was 
clearly equal to or better than 
that which would be expected 
from competent first*tevel super- 
visors. No development needed 
to be recommertded for promo- 
tion. 

GOOD-Performance in the 
Assessment Center was hnvm- 
what bctow that which would b« 
expected from competent first- 
level supervisors. Probably needs 
only supervisory experiertce to 
become a good supervisor* 
would recqnrunend for 
promotion 

FAIR-Ptrformance in the 
Assessment Center was clearly 
below that which would be 
expected from competerit first- ^ 
level supervisors. Development 
needed in some areas before 
recommending (or promotion. 

POOR-Performance in the 
Assessment Center indicated a 
serious lack of ability in many 
areas evaluated. A strong 
development program, followed 
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Final Evaluation Summary— Page 2 



Rating Scale ol 
Rating Candidate Performance 



ORGANIZATION AND LOGIC 
OF ARGUMENTS-Presents 
positJons in a •^cll organized 
fashion and supports them with 
arguments which are appropriate 
and logical and which utilize 
avai>ai;ie information. 



WRITTEN COM^"J^'ICATION 
SKILLS-Demonstiates adequate 
written expression of thougnrs 



DECISION. MAKING SKILLS- 
Judged on whether jny ertoneous 
vr contradictory decisions were 
mjde m the written oxercise» 



UTILIZATION OF SUBORDI- 
NATES-Judged in the written 
exercise by the individual's utiliza* 
tion of subordinates and 
responses to their requests. 



RESPONSES TO SUPERJORS- 
Judqed in the written exercise by 
the responses of the individual to 
the requests and wishes of 
superiors. 



by a re-evaluation is needed 
before recommendation for 
promotion could be considered. 

NO NOT OBSERVED-There was 
not enough behavior exhibited 
in this area :o be able to make a 
scale judgment. 



OVERALL RATING 
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OPPORTUNITY COUNSELING- 
FEEDBACK TO THE CANDIDATE 



• Through the line organuation-normatly the department 
head who recommended him. 

• All feedback must be presented in a constructive manner. 

^ •» 

• The Industrial Relations Staff and Organizational 
Development Department will assist the line organization 
in making recommendations for individual development 
activities. 

• A candidate is allowed to read the overall evaluation report 
and to initial the report to indicate that he has seen it 

• The supervisor who recommended the candidate for 
evaluation is briefed on the candidate's performance 
(usually done by the department head). 

• A signed copy of the evaluation report is returned to the 

Industrial Relations Division. This form i* to inr'itde the - 
employee's reaction to his cvtiluanon and siHScif ic 
development goats agreed upon wtth his supervision. 

• Should the employee not be satisfied with the feedback 
by the line organization, he may request an ^dicnce with 
the Program Coordinator through the line organization for 
more detailed information on his evaluation. 



OPPORTUNITY COUNSELING OBJECTIVES 



• Seeks to combine a complete assessment of abilities using 
simulation techniques and standardized tests along with a 
diagnosis of the candidate's own interests and career gaJs. 

f 

9 Employee centered approach. Emphasis is placed on the 
development of career opportunities for individuals based 
on their own goals and abilities. 
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OPPORTUNITY COUNSELING INTERVIEW 
Obtain individuars reactions to the diagnostic process- 
Obtain indications of ambitions, desires, and interests. 
Review Assessment Center evaluation. 
Discuss other diagnostic instruments. 
Suggest tentative report and recommendations. 
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SUMMARY OF ANALVSISOF UPGRADING PRACTICES 
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Section i 
UPGRADING IN AN INDUSTRIAL SETTING 
SUHHARY OF FINDINGS^ 

rinding productive employees is important to the economic growth of 
any company. Most employees search for opportunities- to advance, others 
are content to remain where they start. A company not sensitive to employee 
concerns will ultimately experience management problems that seriously deter 
its growth and its ability to meet economic objectives. Some companies have 
well established systems for developing employees. However, not much is 
known about what these systems entail. 

This reports some findings of a research effort directed at alialyzing 
the employee upgrading practices and organizational reiiources utilized for 
upgrading in a major industrial firm. The study was conducted to determine 
the impact on employee upgrading of (1) management practices and beliefs 
about employee advancement potential; (2) training programs in the indus- 
trial facility; and (3) collective bargaining agreements and government 
legislation. In addition, the internal roobiJ.ity of the work force was exa- 
mined in order to identify career paths. 

Employee development or upgrading, as it will be called in this sum- 
mary, is especially critical today as a result of changes in the labor 
force. More women are seeking careers, more minorition and other disadvan- 
taged persons have entered the ranks of the employed. Upgrading is also a 
topic of priority since equal employment opportunity legislation requires it. 

Report of Phase I, Part 1, Analysis of Upgrading in an Industrial Setting , 
Oak Ridge Associated Universities, Oak Ridge, Tennessee, available upon ^ 
request. 
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Impact of Management Practices 

Through interviews with managers, the employee promotion process, in- 
cluding employee personal qualities considered important for advancement, 
was described find analyzed. Authority to pi^mote, hire and fire was 
decent rii I izetl^ tiius advancement decisions were made or strongly influenced 
by an employf»o's immediate supervisor. As such the decisions tended to be 
inconsistent bobause supervisors held widely differing opinions about what 
employ u'L? c:haracteristics lead to success, In^fact, a total of 49 different 
cluster;; of characteristics were mentioned wi^.h some frequency by the tirm's 
manager. Tiiosu employee characteristics roost frequently mentioned as im- 
portant included: (1) job knowledge or technical competence; (2) dependa- 
bility, riilidbiiity , and attendance; (3) ability to p^ot alonp, with people; 
and (4) ability lo communicate (written and oral). 

Mori managers did not have detailed knowledge of job openings or 
requirements for jobs outside their* departments. They expressed a definite 
interest in establishing more systematic methods of matching job openings 
with qudJified candidates. 

Another factor affecting the selection process was the availability 
of job o^j^aning information to all potentially qualified candidates. Con- 
sidering ail employees at the foreman level and below, hourly employees had 
the most compretiensi vo system of obtaining job openinf; Information, iargeJy 
as a result of collective bargaining contracts. Weekly and monthly em- 
ployeeii relied on the plant personnel departments, their siq^ervisors , ^nd 
word of mouth for information about job openings. In addition, the weekly 
and monthly employees usually did not have the opportunity to formally 
nominate themscives for a job opening. 



/ 
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In summary, there was a low probability that all potential candidatos 

- ♦ ■ . 

were considered or could nomihatt 'themselves for a position. The likeli- 
hood of non-uniform selection criteria being applied in promotional deci- 
sions was hic^h. There were few objective means of evaluation to supplement 
supervi'-^ory judgment. All of these factors reduced the company's ability 
to provide equal opportunity for all its employees. 

Impact of Training Programs 

Employees responsible for training functions ^ both line managers and 
training personnel, were surveyed to determine what part formal or informal 
training programs had played in employee upgrading. 

When the identified programs were categorized by purpose, "por»ition 
training" or specialized training accounted for the majority of tlio pro>7.rcims. 
The general intent of these programs was to teach, or update job skills 
needed by employees to maintain current jobs rather than to develop skills 
for higher level jobs. Most supervisory training programs were also designed 
for employees already promoted to the supervisory ranks, rather than for 
presupervisory training of employees. 

There were educational assistance and university study programs that 
were directed toward upgrading. Additionally, some divisions and department 
used job rotation to develop employee versatility. This gave the employoe 
instruction in what jobs were available within the department and the skills 
needed to perform them. The education obtained by these formal and informal 
means probably increased chances for promotion. 

impact of Legislation and Collective Bargaining Agreements 
National legislation and executive orders concerning nondiscrimination 
in employment have received, wide coverage and need not be summarized here. 



That this legislation has had a strong impact on company practices is 
illustrated by the fact that black worKdrs approximately tripled their 
representation in the work force and. ix the number of jobs at which they 
worked* I 

Seven different contracts, invoLving, fbifr different unions,, were in 
effect at any one time during the study. In response to legislation all 
collective bargaining agreements added. an identical nondiscrimination 
clause: "Both the company and the union agree that the provisions of tliis 
contract shall be applied to all employees without regard to any indivi- 
dual's race, color, religion, sex or national origin." Additionally*, appren- 
ticeship standards prohibit discrimination on the- basis of "occupational ly 
irrelevant physical requirements." 

As mentioned earlier, hourly employees (nonrexempt) had a comprehen- 
sive, system for obtaining jpb opening- information. Collective bar^'.aininp. 
,v:^,'<?ontracts required posting publicly the ho.urJ,y job openings, pay scale, 
dei;cri|Jtion and qualifications. Promotion opportunities were largely pre- 
determined by contract, and seniority provisions. Thus, hourly workers , 
jjarticularly those in craft, apprenticeship programs were provided" with 
well defined, career paths . 

Another finding of the. contract, analysis: was restraint on downward 
job bidding. This tended to limit an employee in. the positions he could 
move to, possibly eliminating movement into: positions with more promotion 
potential. 

Two different seniority systems were- in. effect at different sites in 
this company. Under one: system,, an ^ruployee.'s. seniority, was determined by 
date of hire, and. this- seniority, remained, with the employee as he was 
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promoted from one job grouping to another. In the other system, seniority 

was determined by the employee's length of service within a job; chanj'inp, 

from one seniority grouping to another, he lost his seniority in the former 

grouping and had to accumulate seniority in the new grouping. This may 

have had an inhibiting effect on job upgrading. . Due to the provisions for » 

"bumping** employees with lower seniority whenever layoffs are necessary, 

an employee would be hesitant about relinquishing even a small amount of , 

seniority. 

In general, collective bargaining contracts neither favored nor dir;- 
favored upgrading. Although they had substantial impact on upgradinj', , the 
impact was not consistent. 

I- 

Impact of Employee Mobility 
Analysis of employment records over an eight-year period for jobs below 
and including first level foreman demonstrated clearly that considerable 
employee upgrading had taken place. Craft occupations, particularly the 
machine trades, had the highest upgrading rate (as was expected). It also 
showed that managerial type jobs were very slow in movement, while clerical 
and tectinical occupations had slow movement jobs, but also many high move- 
ment jobs. 

The search for career paths was not very fruitful. The company had no 
growth and extremoly low turnover (about 7%), and tho dlroction of upward 
movement was diffuse — ^not predictable from an incumbent's previous position. 
Further, a large number of jobs had only a few individuals in each. The 
small amount of movement in these jobs made it impossible to identify career 
paths among them. The result of this combination of facts was that only a 
few career paths were identified, and most of those that did exist were 
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defined by union contracts. Some others were in clerical aroas tiiid riiiinn^> 
laboratory technicians. 

Another factor was the apparent existence of some conflict betwor?n 
direct hiring and in-plant promotion (a notable exception being foreman 
level jobs where 92% were filled through internal promotion). Also, entry 
port jobs were not always at the "lowest practical level." 

To sum up, despite the existence of considerable employee mobility, 
this movement was hard to define due to the diffuse nature of the movement 
and factors associated with the organizational structure. 

The firm studied had an employee upgrading system that placed rosf»on- 
sibility for identifying advancement potential on first-line supervinorr; . 
Tlie company had a written policy stating that promotion to higher loweJ 
positions would occur whenever possible. Limited by tho occuprit Lonal r:truc- 
tures and diversity of technician jobs not many careor j^.-Uh^; existed, y(»t 
promotions occurred. Specific preparation of empJoyoor, lor^ hifjior lev^.l 
jobs occurred through on-the-job training, while educatioridi a::isii;t.ince 
supported broader employee development goals, Tho element of the up;;.radi nj'. 
system that the company desired to strengthen was the identif icdtion pro- 
cess of employee potential. 
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Upgrading— An Assessment Method for Enlarging the Pool of 
Eligible Technicians for Supervisory Positions^ 

Selection of first-line supervisors has and is accomplished using a 
range of approaches from the "I dub thee supervisor" (because I like you) 
technique all the way to the "multiple hurdle testing" technique (clinical 
interviews, paper-pencil tests). More often than not the "I dub three" 
technique is predominant. This report describes the results of an experi- 
mental use of the selection approach that falls somewhere between these 
two, and the usefulness of the approach as an employee upgrading tool. 

Assessment Center 

The selection approach tested is the assessment center which consti- 
tutes simulations of selected work-related problems encountered in supervi- 
sory jobs and measuring each candidate's responses to these prohlnm:; . Tho 
simulations include group discussion and group and individual problom 
solving activity. A management panel observes and scores individual be- 
havior during a full day of candidate interaction. 

Opportunity Counseling 

The observations and scores of candidate behavior are reported to each 
candidate in an individual counseling session along with aptitude and in- 
terest test scores. During the sessidS,' general and personal occupational 
opportunities are discussed, and .plans for individual development are 
explored. 

Experimentation with the assessment center and opportunity coiins».>l ir.)* 
was undertaken in order to increase the size of the pool from whicii new 
supervisors are selected, to make the selection process more objective, to 
communicate job and career prospects to employees, and to formulate employee 
development alternatives. 



^Report of Phase I, Part 2, Upgrading in an Industrial Setting, Oak Ridp.e 
Associated Universities, Oak Ridge, Tennessee, available lapon request. 
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Assessment Center and Opportunity Counseling Evaluation 
The assessment center and opportunity counseling have found ready 
acceptance. Apparently, management's need to make good promc>tior>-;l deci- 
sions, and the need of the employees' to make sound career decisions made 
both groups receptive to a means of obtaining more objective information. 
The evaluation of the assessment center vras accomplished by surveying the 
people it serves (managers, stipervisors and participants) and by examining 
the interrelationships of the different exercises and dimensions to deter- 
mine whether they correspond to each other in reasonable ways. Hanaigers 
were asked to rate the frequency with which the center provided use .'ul in- 
formation. The average rating was three, which meant that- the center usually 
provided useful information on a four-point scale. No respondent checked 
lowest of the four categories. The ways in which this information is per- 
ceived and put to use by managers is best siimmarized in their own comments. 
"It gives the supervisor an independent objective view of the candidate, 
helps the candidate realize his strong and wea^ points, the employee is 
given an outside evaluation of his skills, prevents political promotion." 
A slight majority of the managers felt that the center's results were use- 
ful primarily in the area of making promotional decisions, while a slightly 
smaller number perceived results as most useful in employee development. 
Approximately^ half of the managers stated that assessment center results 
would be a strong factor in determining promotion, but not the major one. 
Ninety percent of the employees who parldcipated in the assessment center 
believed the process to be fair, and 57% felt that it allowed a chance to 
show their real capabilities for supervisory positions. The judgments of 
managers and participants support the voiiue of the assessment center pro- 
cess. First-line supervisors were sligh^y less enthusiastic because they 
were more impressed with occasional disagreements with assessment results; 
however > they too found it to be a sovxrce of information not otherwise 
available. 

All three groups — managers, supervisoz^, participants — made comments 
showing insight into the assessment Center process. These comments are 
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expected to serve an important role in further development of t.lie fir.^erjn- 

menl center. An analysis of the scoring procedures and dimensions on which 

oehavior was t^ated revealed a high degree of interrelatedness or correlnt ion. 

It was concluded that the complexity of the dimensions accounted for a p.roat 

deal of the correlation and that revision and simplification of the dimon- f 

sions could make them more distinct and useful. 

Paper and pencil tests provided additional information that was essen- f 
tial in opportunity counseling* Aptitudes and interests of each employnf? 
were, on the basis of their judgments, quite accurately reflected, and their 
systematic measurement made a discussion of career goals and career planning 
more easily conducted. 

Opportunity counseling, which was wholly voluntary, was very wcvl 1 re- 
ceived. Participants showed great interest in the process, and it cipparc?ntly 
stimulated serious discussion of personal career plans. It also p,ave ^'m- 
ployees an opportunity to put into a formal record relevant career informa- 
tion. Once the instruments used in the counseling process (assessment 
center, Jiandardized tests, and interviews) have been thoroughly validrit#*d, 
a sound basis will exist for further development of specific training 
programs . 
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